The team managed to find these new metals through a combination of AI and lab experiments. First, they had to overcome a significant challenge: a lack of existing data they could use to train the machine-learning models. They trained the models on the data they had--several hundred data points describing the properties of existing metal alloys. The AI system used that data to make predictions for new metals that would exhibit low invar. The researchers then created those metals in a lab, measured the results, and fed those results back into the machine-learning model.

Common Crawl is an open repository of web crawl data. The researchers used English language documents which have long paragraphs because they wanted a data which allowed modelling of long range dependencies. The researchers constructed batches of 32 word pieces. To begin with, the goal was to determine the maximum number of GPU workers which can be employed for SGD. The researchers also tried asynchronous SGD with 32 and 128 workers and found that with large number of workers it is difficult to keep training stable.